Custom Rag Applications

Overview

This document describes how to integrate "Custom RAG Applications" with DynamoEval.

The CustomRagDB wrapper by DynamoAI facilitates integration with your vector database through REST APIs. Key features include:

• Acting as a bridge between DynamoEval and custom vector stores
• Integrate with vector databases using REST APIs
• Support for custom retrieval routes

Requirements

1. Your custom RAG application must have at least one route of type "retrieve"
2. The application must handle authentication if required (supported types: NO_AUTH, BEARER)
3. The application must either:
   • Accept DynamoEval's standard request format and return the expected response format, OR
   • Use JSONata transformations to convert between your format and DynamoEval's format.

JSONata Transformations

Use JSONata to translate between DynamoEval's RAG request/response formats and your application's API.

• Request transformation: Map fields like query and top_k into your API's expected request shape.
• Response transformation: Map your API's retrieval results back into DynamoEval's expected retrieved and text_only fields.

See "Example 2" below for concrete request/response transformation expressions.

Request/Response Contract

Request Format

DynamoEval's standard request format:

{
    "query": "user query text",
    "top_k": 3
}

Response Format

DynamoEval's expected response format:

{
    "retrieved": [
        {
            "id": "doc1",
            "score": 0.95,
            "content": "document text - 1"
        },
        {
            "id": "doc2",
            "score": 0.75,
            "content": "document text - 2"
        },
        {
            "id": "doc3",
            "score": 0.89,
            "content": "document text - 3"
        }
    ],
    "text_only": ["document text - 1", "document text - 2", "document text - 3"]
}

Code Snippets

Create Custom RAG Application

Example 1: Direct Integration (with No Authentication)

If your API already matches DynamoEval's format and doesn't require authentication:

from dynamofl.entities import CustomRagApplicationRoutesEntity, AuthTypeEnum, RouteTypeEnum

base_url = "https://api.example.com"
auth_type = AuthTypeEnum.NO_AUTH

custom_rag_application_routes = [
    CustomRagApplicationRoutesEntity(
        route_type=RouteTypeEnum.RETRIEVE,
        route_path="/retrieve",
        request_transformation_expression=None,
        response_transformation_expression=None,
    )
]

dfl.create_custom_rag_application(
    base_url=base_url,
    auth_type=auth_type,
    auth_config=None,
    custom_rag_application_routes=custom_rag_application_routes
)

Example 2: With Transformations (with Bearer Token Authentication)

If your API uses a different format, you'll need to use JSONata transformations. For details about JSONata transformations in DynamoEval, refer to the JSONata Transformations section in Custom Applications documentation.

# Your API's format:
# Request: {"searchQuery": "text", "limit": 3}
# Response: {"results": [{"docId": "1", "relevance": 0.9, "text": "content"}]}
from dynamofl.entities import CustomRagApplicationRoutesEntity, AuthTypeEnum, RouteTypeEnum

base_url = "https://api.example.com"
auth_type = AuthTypeEnum.BEARER
auth_config = {"token": "your-bearer-token"}

# Transform DynamoEval format to your API format
request_transformation_expression = """
/* Convert DynamoEval's expected format to custom format */
{
    "searchQuery": query,  // Maps DynamoEval's "query" to your API's "searchQuery"
    "limit": top_k        // Maps DynamoEval's "top_k" to your API's "limit"
}"""

# Transform your API response to DynamoEval format
response_transformation_expression = """
/* Convert custom API response to DynamoEval's expected format */
{
    "retrieved": results.{
        "id": docId,      // Maps your API's "docId" to DynamoEval's "id"
        "score": relevance, // Maps your API's "relevance" to DynamoEval's "score"
        "content": text    // Maps your API's "text" to DynamoEval's "content"
    },
    /* Create array of just the content */
    "text_only": [results.text]  
}"""
custom_rag_application_routes = [
    CustomRagApplicationRoutesEntity(
        route_type=RouteTypeEnum.RETRIEVE,
        route_path="/retrieve",
        request_transformation_expression=request_transformation_expression,
        response_transformation_expression=response_transformation_expression,
    )
]

dfl.create_custom_rag_application(
    base_url=base_url,
    auth_type=auth_type,
    auth_config=auth_config,
    custom_rag_application_routes=custom_rag_application_routes
)

Supported Authentication Types

Custom RAG Applications support the following authentication types:

1. No Authentication

from dynamofl.entities import AuthTypeEnum

auth_type = AuthTypeEnum.NO_AUTH
auth_config = None

2. Bearer Token

from dynamofl.entities import AuthTypeEnum

auth_type = AuthTypeEnum.BEARER
auth_config = {"token": "your-bearer-token"}

All authenticated requests will include the following default headers:

{
    "Content-Type": "application/json;charset=UTF-8",
    "Accept": "application/json, text/plain, */*"
}

Authentication Header Examples

The authentication headers will be generated as follows:

• Bearer Token: Adds "Authorization": "Bearer your-bearer-token"
• No Auth: No additional headers added